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Abstract 

We consider a zero sum differential game with lack of observation on 
one side. The initial state of the system is drawn at random according 
to some probability /iq on TR N . Player I is informed of the initial 
position of state while player II knows only fig. Moreover Player I 
observes Player IPs moves while Player II is blind and has no further 
information. We prove that in this game with a terminal payoff the 
value exists and is characterized as the unique viscosity solution of 
some Hamilton- Jacobi equation on a space of probability measures. 
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Introduction 

We consider a two player zero sum differential game in TR with finite hori- 
zon T > 0. Its dynamics is given by : 

x'{t) = f{x(t),u(t),v{t)) , t G [t ,T], u(t) G U, v{t) G V _ 
x(t ) = x 1 

where Player I uses the measurable control u G U(to) := Z^Qto, T], U) and 
Player II the measurable control v G V(io) := L l {[to,T], V). We denote by 
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(X tomv ) the solution of (1), which is unique under suitable assumptions 
on / stated below. In this zero sum game, Player I aims at minimizing a 
final cost g(x(T)), where g : 1R N — > K, while player II aims at maximizing 
it. We introduce lack of observation in the following way: 

• At time to, the initial state of the system, xo, is drawn at random 
according to some probability measure [1q on IR^; 

• Player I is informed of x$ while Player II is only informed of ^q; 

• During the game, Player II observes neither the state of the system 
and nor the control played by his or her opponent, while Player I has a 
full information on the control played so far by Player II (and therefore 
on the state of the system as well). 

Our aim is to prove that the game has a value and to characterize this 
value as the unique viscosity solution of some Hamilton-Jacobi equation. 
Since the natural state of the system is the space of probability measures 
on ]R, N , this Hamilton-Jacobi equation takes place in this space. 

Games with asymmetric information were studied — mostly on examples — 
by several authors: see for instance Bernhard and Rapaport [2], Gal [8], 
Petrosjan [11] and Baras and James [1]. In this latter reference the au- 
thors introduce an underlying Hamilton-Jacobi equation in some infinite 
dimensional space, but, since the game is seen as a control problem with 
disturbance, the value function considered there differs considerably from 
ours. 

Our game has actually much to do with a previous work of Cardaliaguet 
and Quincampoix [3] which analyses problems in which the only informa- 
tion that both players have on the initial position of the system is that it 
has been randomly choosen according to some probability known to both 
players. In [3], the players observe each other. This is a main difference 
with our problem, where the lack of observation of one player induces the 
use of completely asymmetric strategies. The introduction of a suitable no- 
tion of strategies to formalize this situation is one of the novelties of our 
paper. A dramatic consequence of the asymmetry of information is that 
the usual machinery of differential games (dynamic programming, which 
leads to the characterization of the value functions as the unique solution 
of some Hamilton-Jacobi equation) does not work. Indeed the lower value 
does not seem to satisfy any dynamic programming, because the uninformed 
player cannot actualize his or her strategy along the game since he or she 
sees nothing. However, and fortunately, it turns out that, in the game for 
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the upper value, the uninformed player, knowing the strategy of his or her 
oponent, can actualize his or her own strategy along the time. This leads 
to a dynamic programming for the upper value, which takes place in the 
space of probability measures on JR N . From this we derive that the upper 
value satisfies a Hamilton-Jacobi equation in some suitable viscosity sense. 
The definition of the viscosity solution in this framework is inspired by, but 
slightly differs from, the one given in [3]. Other definitions of viscosity solu- 
tion in the Wasserstein space have been used in the literature, in general for 
more singular dynamics (see for instance [6, 7, 9, 10]). Then the existence 
of a value (i.e., the fact that the upper value coincides with the lower one) 
relies on min-max arguments combined with PDE ones: we introduce an 
auxiliary game in which the uninformed player chooses a strategy by ran- 
domizing over a finite set of controls. Existence of a value for this game is 
obtained by min-max arguments. Then we show that this auxiliary game 
is close to the continuous one by using techniques from Crandall and Lions 
[5] on the stability of viscosity solutions for Hamilton-Jacobi equations in 
infinite dimension. 

The paper is organized in the following way: in the first section, we define 
the strategies and state the assumptions on the game. In the second section, 
we prove that the upper and lower value functions are Lipschitz continuous. 
In the third section, we state the dynamic programming principle for the 
upper value function. In section 4, we show that if a function satisfies this 
dynamic programming principle, then it is the unique viscosity solution of 
some Hamilton-Jacobi equation. In section 5, we introduce some discrete 
approximation for the game. In the last section, we prove, using the discrete 
game, that the game has a value. 



1 Definitions and assumptions 

We first introduce some notations on the space of probability measures on 
TR N . For a fixed closed subset K of TR N we denote by W(K) the set of Borel 
probability measures with support included in K and with finite second order 
moment. We set W = W(TR N ). For any fi, u G W, let n(/x, v) be the set of 
probability measures on JR 2N with first marginal [i and second marginal v. 
Recall that the Wasserstein distance on W between n and v is defined as 

d 2 (fi,u) = inf / \x — y\ 2 d7r(x, y) . 
Treno^) J U 2N 

It is well-known that this infimum is in fact a minimum and we denote by 
IIopt(/^, v) the set of minimizers in the above minimization problem. If [i 
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is a probability measure on a set X and <p : X — > Y, we denote by ipftfi 
the pushforward image of fi by ip, denned by tp$/ji(A) = fi((p~ 1 (A)) for any 
subset A of Y for which this definition makes sense. 

Next we introduce notations and assumptions related to the game. The 
payoff only depends on the terminal state of the system. More precisely, if 
at time T the system is at some position x(T), then the outcome of the game 
is g(x(T)), where g : JR N ->• IR is a fixed Lipschitz continuous and bounded 
function. Assume that the initial state is choosen at random according to a 
probability measure /j,q G VV at the initial time to and suppose for a while 
that the players use a pair of controls (u,v) G U(to) x V(to) independent of 
the initial state. Then the outcome of the game is 



Jtr n 

Throughout this paper we tacitely assume that the following conditions on 
the data are satisfied: 

i) U and V are compact subsets of some finite dimensional vector spaces, 

ii) f is bounded, uniformly continuous on IR^ x U X V, 

and uniformly Lipschitz continuous with respect to the x variable, 
Hi) g is Lipschitz continuous and bounded. 



During the proofs, we denote by C a generic constant depending on N, f 
and g. 

For any < to < t\ < T we denote by U(to,ti) the set of Lebesgue 
measurable maps u : [£o>*i] ~~ > U. We abbreviate the notation into U(to) 
whenever t\ = T. We endow U(to, t\) with the L 1 distance 



and with the Borel a— algebra associated with this distance. Recall that 
U(to,ti) is then a Polish space (i.e., a complete separable metric space). We 
denote by A(U(to,ti)) the set of Borel probability measures on U(to,ti). 
This set is endowed with the weak-* topology, for which there is an associ- 
ated distance defined as follows: 



where the supremum is taken over the set of Lipschitz continuous maps 
ip : U(to, ti) — ^ [—1, 1] with a Lipschitz constant less than 1. 
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The sets V{to, ii) and V(io) of Lebesgue measurable maps v : [to, t\] — > V 
and v : [to,T] —> V are denned in a symmetric way and endowed with 
the L 1 distance and with the associate Borel a— algebra. The set of Borel 
probability measures on V{to,t±) is denoted by A{V{to,t±)). 

We say that a map (x,v) P v x from M N x V{t ) into A{U{t )) is 
measurable if, for any Borel subset A oiU{to), the mapping (x,v) — > Px(A) 
is Borel measurable. 

Definition 1.1. A strategy for Player I for the initial time to G [0,T] is 
a measurable mapping (x,v) — > P x from IR^ x V(io) into A{U{to)) which 
fulfills the following nonanticipativity condition: there is some delay r > 
such that, if two controls V\,V2 G V(*o) coincide a.e. on [to,t] for some 
t G [to,T] and if Rt At+r)/\T denotes the restriction mapping from U{to) 
onto U {t , (t+r)AT), then the measures R to ^ t+T - )AT ^ 1 and R to ^ t+T)AT $P£ 2 
coincide (on U{to, {t + r) A T)) for any x € TR N . 

Note that strategies for Player I actually correspond to behavioral strate- 
gies in game theory, because Player I adapts his or her probability measure 
in function of the past behaviour of his or her oponent. The heuristic inter- 
pretation of a strategy P is that, if the state of the system is at the initial 
position x, then Player I answers (in a nonanticipative way) to a control 
v G V{to) played by Player II a control u G U{to) with probability P%{u). 

We denote by A{A x {to)) the set of strategies for Player I, by A{A 1 x {to)) 
the set of such strategies which have a delay r and by A^ito) the subset 
of A{A T £ {to)) consisting in deterministic strategies, i.e., strategies for which, 
for any \x,v) G JR N x V(t ), P£ is a Dirac mass. If P G A T £ {t ), then 
there is a map a : TR N x V{to) — > U{to) such that dP^{u) = dS a / XtV \{u), 
and this map satisfies the nonanticipative property: for [1q— a.e. x G TR N , if 
two controls v±,V2 G V{to) coincide a.e. in [to,t] for some t G [to^], then 
a{x,v±) = a{x,V2) a.e. in [to, {t + r) A T}. Generic elements of A T x (to) are 
systematically identified with the maps a. 

Since Player II observes neither the state nor his or her oponent behavior, 
the definition of his or her strategies is much simpler than for Player I: 

Definition 1.2. A strategy for Player II is a Borel probability measure Q 
on the set V(to)- 

Recall that we denote by A{V{to)) the set of such strategies. Given 
Q G A(V(io)) an( i P £ A{A x {to)) we denote by Z{to, A*o> P, Q) the outcome 
of the two strategies P and Q: 

3{t ,Li ,P,Q) = [ 9 dP v x {u)dQ{v)dno{x) . 

^IR JV 'xW(to)xV(to) V 7 
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We are now ready to define the value functions. The lower value of the game 
is: 

V-(t ,n ) = lim sup inf 3 (to, Mo, P, Q) 

r^o+ QeA(v{ t Q)) PeA(Ai(t )) 

= lim V~(t ,/j,o) 

where we have set 

V"-(i ,Mo) = sup inf J3(t ,fj, ,P,Q) 

QeA(V(t )) PeA(Ai(t )) 

= sup inf Z(to,HQ,a,Q) 



(3) 



The upper value of the game is defined in a symmetrical way: 

V + (£ ,W)) = hm inf sup 3(*o, fio, P,Q) 

r^o peA(AUto)) QeA(v(t Q )) 

= ^l n f . SU P 3(to,Vo,P,v) 
PeA(A x (t ,no)) vev(t ) 

2 Regularity of the value functions 

We begin by proving the Lipschitz continuity of the upper and lower value 
functions, which is important for the characterization of the value as a vis- 
cosity solution of some Hamilton-Jacobi equation. 

Proposition 2.1 (Regularity of the value functions). The value functions 
V + and V~ are Lipschitz continuous on [0,T] x W. 

Proof. We start with the Lipschitz continuity of V + with respect to the /i 
variable. Let to G [0, T], fi, v G W and choose 7 G H opt (v, fi) some optimal 
transport plan between [i and v. Let us recall that 7 admits a desintegration 
of the form d"f(x,y) = d^ x (y)dv(x) where the map x — > 7^ is measurable, 
i.e., such that the map x — > 'Yx(A) is Borel measurable for any Borel set 
A C M N . Let P G A(^(t )) be an e-optimal strategy for V + (t ,fJ,), i.e., P 
satisfies 

sup 3(t ,fi,P, v) < V + (t ,/u) + e . 

t>eV(t ) 

We define the strategy P G A(^ x (to)) by 

/ <p(u)dP v x (u) = f v(u)dP v y (u)d lx (y) 
JU{t ) JTR N xU(t ) 
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for any (x, v) G JR N x V(to) and f° r any nonnegative Borel measurable map 
(p : U(to) — > K. Let now v G V(io) and let us estimate 3(to, v, P, v): we have 

Z(to,u,P,v) = I g (X^ v ) dP v y {u)d lx {y)dv{x) 



< 



g (X$>"> v > v ) + C\x - y\] dP^(u)d 7 (x,y) 



IU 2N xU(t ) 

< I g(x^ n ' v )dP^(u)d^y) + C I \x-y\d 7 (x,y) 



'K xW(to) 

< V+(t ,fi) + e + Cd(fi,u) 

Therefore 

V+(t ,^)< sup 3(t Q ,v,P,v) < V + {t ,fi) + e + Cd(n,u) . 

vev(t ) 

This proves the Lipschitz continuity of V + with respect to second variable, 
uniformly with respect to the time variable. 

We now prove that V + is Lipschitz continuous with respect to the time 
variable. Fix to < t\ < T, fi G W and vq G V(to). We choose some e-optimal 
strategy P for Player I in V + (io, n) and define the strategy P G A(A x (ti)) 
by: 

/ V (ui)d^(ui)= / v(u, )dP^ v \u) 
Ju(ti) Ju(to) l1 ' 

where (i>o,f) denotes the concatenation of the controls vq and u, for any 

(x, v) € JR N x V(ti) and any nonnegative Borel measurable map <p : U(t\) — > 

IR. Then, for any v G V(ii), we have 

3(ti,A*,P,w) = / g (xt^^) dPt> v) (u)dv(x) 

JM N xU(t ) \ / 

< / [5 (x^'^) + C(ti - t )l («)d M (x) 
Jm. N xu(to) 1 v y J 

< V + (t ,M) + e + C(ti-to) 

Therefore we get: 

V+(ti, n) - V + (t , m) < e + C(ti - to) • 

For the reverse inequality, let P G A(A x {ti)) be some e-optimal strategy 
for player I in V + (ii,/z). We fix uq G ZY(io) and define the strategy P G 
A(A(to)) by 

/ <p(u)dPZ(u)= I ¥»((«o,«i))dPx' [tl,I1 («i) 
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for any (x,v) G JR N x V(to) and for any nonnegative Borel measurable map 
(p : U(to) — > Hi. Then, for any v G V(io), we have 



< 



TR"xU(ti) 



R Jv xW(ti) 



f v to,:r,(«0,«l),f 
5 ( A T 



[tl.T] 



(ui)dfi(x) 



ff|X T [tl ' T] ) +C(ti-t ) 



dp*' 1 ' 1 ' 71 («i)d/i(x) 



Hence: 



< V + (ti,/i) + e + C(ti-to) 



V+(t ,^)-V + (t 1 , / u) <e + C(ti-t ) • 



which shows that V + is Lipschitz continuous with respect to the time vari- 
able, uniformly with respect to the ji variable, since e is arbitrary. 



The proof of the Lipschitz continuity for V goes along the same lines, 
so we omit it. □ 



3 Dynamic programming for the upper value func- 
tion 

We prove in this section that V + satisfies some dynamic programming prin- 
ciple. We have to define how Player II's information evolves in time. In 
the game V + (io,/Uo), Player II knows the initial distribution of the state 
variable as well as his or her opponent's strategy P. If he or she plays the 
control v G V(to), his or her information on the state of the system at time 
t\ G (to,T] is the probability measure ^^ 0,P)V defined by: 

V<peC b QR N ,M), [ p(x)drf°^> P > v (x) = [ v{Xll' x ^ v )dPV{u)dM{x) 
Note that $ 1 > IH> > P ' V belongs to W. 

Proposition 3.1 (Dynamic programming principle for V + ). For any (to, h,no) 
such that t\ G (to,T], we have: 

V+(t , M0 )= inf sup V+Cti.Aig^). 

P6A(A=(to)) we v(to) 

Proof. We denote by W(to, ti,no) the right-hand side of the previous equal- 
ity. Arguing as for Proposition 2.1, one can show that W is Lipschitz con- 
tinuous with respect to the measure variable. 
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Let us now show that we can assume in addition to (2) that / has a uni- 
formly bounded C 2 norm with respect to the x variable. Indeed, from our 
assumptions on /, if we mollify / with respect to the x variable, we obtain a 
sequence of uniformly continuous functions f n : JR N x U x V — > JR N , with a 
modulus of continuity independent of n, uniformly (with respect to n) Lips- 
chitz continuous in space and which converge uniformly to / on x U x V. 
We easily check that the upper value function V+ for f n corresponding to 
f n converges to V + and that the W n converge to W uniformly on [0, T] x W. 
We also note that the transported measures ^ '^°' P,V for f n converges in 
W to fi t °' fl0 ' P ' v uniformly with respect to P and v. So, if Lemma 3.1 holds 
for the f n , it also holds for /. Therefore we can assume, from now on, that 
/ has a uniformly bounded C 2 norm with respect to the x variable. 

Let us first prove that V + (io,Mo) < W(*o>*ij Mo) under the additional 
assumptions that po € W(K), where K is some compact in JR N . This extra 
assumption is removed later. The first step consists in regularizing pq. Let 
p € C%°(TR, N ) be a smooth mollificr: p > is even, has a support in the unit 
ball and satisfies J^jv p{x)dx = 1. Let p £ (x) = e N p(j), fe = Pe* Po and 
p e = f e dx. By standard arguments, we have that d(po,p e ) < e. 

Lemma 3.2. For any strategy P € A(A x (to)), there exists a strategy P e <G 
A(A x (to)) , with the same delay as P, a compact set K\ C JR N and a con- 
stant C such that, for any v G V(io) an d an V t G [to,T], 

1. d(p\ ^\p\°'^' v ') <Ce, 

2. [j t t ' fJie > Pe > v fi as a support in K\ and a density /"' e bounded in C l {Ki) 
byC, 

Proof. Let P e G A x (to) be defined by: if f e {x) > 0, then we set 

/ ^(u)dPl x (u) = [ V(u)p e {x - y)dP;(u)dp (y) 

JU(to) Je{x) JjR N xU(t ) 

for any (x, v) € IR^ x V(to) an d for any nonnegative Borel measurable map 
if : U(t ) ->• H. If f e (x) = 0, we just set dP" x (u) = dP^{u). 

Since p e has bounded support and the dynamics is bounded, there is a 
compact set K\ such that ^ t t ' fJ-e ' Pe,v has a support contained in K\ for any 
v € V(to) and any t £ [to, T\. 
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We now compare /j t t ^°' P ' v to p^^' P ^ v for any v € V(t ) and any t € 
[to ,T]: we have 



./IF 



R 2Jv xW(t„) 



■gto,x,u,v _ j£-to,y,u,v 



p e (y-x)dP^(u)dp (x)dy 



because the probability measure 7 on IR^ V defined by 

/ <p{x,y)dr 1 (x,y) = [ V (X^\X^' v ' u '%S-x)dIi;(u)d^(x)dy 
Ju 2N Jm. 2N xU(t ) 



m 2N xu(to) 

satisfies 7 € U{p^°' pv , $> li " p " v ). Since 
we get 

d 2 (fi t t 0,tt0 ' p ' v ^^ t ' Pt ' v ) 



< C\x-y\, 



R iiv xW(to) 



|z - y|V(z/ - x)dP^(u)dp (x)dy 



<Ce 2 



TR 2N xW(to) 



p e (y - x)dP^(u)dfi (x)dy < Ce 2 



Therefore for all v G V(to) and any i € [io>^1 : 

d(/i^ ' P '", $'> t « p *< u ) < Ce . 

We now check that the measure ^^ e ' Pt ' v is absolutely continuous and has 
a density bounded in C l {K\) uniformly with respect to v and t. We first 
note that for fixed (t,u,v) € [£o>^1 x W(io) x V(io) the map T(t,u,v) : a; 1— > 
j£to,x,u,v - g Q £ c j ags ^2 w ^h a £2 i nverse because the dynamics / is of class 

C 2 with respect to the x variable. We denote by T(i, u, this inverse. 
We have, for all ip £ C^(TR N ,JR): 



IB- 



<p{x)d$'>*' p *> v {x) 



IR 2JV xW(to) 



^ (x t ( ^,«,, )pe(x _ y )dP V {u)dliQ{y)dx 



= / ip{z)p £ (T(t,u,v) 1 (z) -y)\detJ T , t v) -i(z)\dP£(u)diJ,o(y) dz 

Therefore ^^ e ' Pt ' v i s absolutely continuous with a density given by 



ft 



p e (T(t,u,v) l {z) - y)\det J Tit ^ v) -i(z)\dP£ (u)d/i {y) . 



xW(to) 
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Note that /"' e is bounded in C 1 , uniformly with respect to v and t, thanks 
to our assumptions on the dynamics /. □ 

We now proceed in the proof of inequality V + (io,/^o) < W(to,ti, no) 
under the additional assumption that /j,q G W(K). Let Pq G A(A x (to)) be 
an e— optimal strategy for W(to, t\, fio) and P e G A(A x (to)) be the strategy 
associated to Po as in Lemma 3.2. 

We first note that P e is Ce— optimal for W(to, t\, /x e ): indeed we have 

sup V + (t 1 ,$ 1 ' ,t " P « v )< sup V+(t 1 ,^' w, ' Po ' ,; ) + C'd(^°' /l£ ' Pe '",^0'' 10 ' Po ' ,; ) 
•uev(to) vev(t ) 

< W(t M,»o) + Ce < W(to,t u iJi e ) + Ce. 

For v G V(to) and t £ [*o>71j let /t^ be the density of the measure 
^,txe,Pz,v_ We denote by F the c i osure in L l {M N ) of the set {/^' £ , v G 

V(to) 5 i G [^0) P]}- Since, from Lemma 3.2, the elements of F have a support 
contained in a fixed compact set K\ and are uniformly bounded in C 1 , F is 
a compact subset of L 1 (K Ar ). Therefore, for any fixed r\ > 0, we can find 
a partition (Oj)j=i,...,n of F into Borel (for the L 1 — topology) subsets with 
a diameter in L 1 less than rj. Let /j G Oj, ft = /jcfa and P« € A( < A a; (fi)) 
be an (e/6)— optimal strategy for V + (ti,ft). Let us check that, if rj is 
small enough, then the strategy Pi is still e/2— optimal for V + (ii,^) for any 
measure ji G F such that — < rj, where is the density of ji. 
Indeed, for all v G V(ii), we have 



< 



g{X ^ ){Ux) _ K{x)) d pv {u)dx 

TR N xU(t ) 

< \\g\\oo\\hp - fi\\ L i < v\\g\\oo < e/6. 



So 



sup Z(h,fJ.,Pi,v) < sup Z(h,fJ,i,Pi,v) + e/6 

t>€V(ti) i>€V(t ) 

< V+(ti,ft) + e/3 < V+(ti,/i) + e/2. 

Let r be a common delay for P e and for all the Pj (i = 1, . . . ,/). For 
v G V(t ), we set ji\ = (i t °'^' Pe ' v . Since W(i ) = W(i ,*i) x we can 

write any u € W(io) as u = (ui,«2) where ui G U(to,t±) and it2 G U(t±). We 
define the strategy P G A(«A(t )) by 



'W(to) 



(p(u)dP^(u) = V l„« e Oi / ¥>((tn,u 2 ))dP. K.m.v^Jdi^x^i) 

j =1 JU(to,tl)xU(tl) l ' A tl -r 
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(where, with a slight abuse of notation, P£ x still denotes the natural restric- 
tion of the measure P£ x to U(to,t\)) for any (x,v) € JR N x V(io) and any 
nonnegative Borel measurable map 92 : U[to) — > IR. Then 



J 



xW(t ,ti)xW(ti) 



\ If I 

,T] dP. ' t t 1 T J,„ 1 , t ,(M 2 )(iP ( ! ; :E (tii)(i// e 



i=i yR 
< J^MeOi / 

1 </IF 



ti,y,U2,v 



ti,X. 



tr\, X.U-i ,V 



g[x, 



1 4*1, 



r] 



T 



dP. ^o^^j^ (u 2 )dP e " x (ui)d/i e (x) 



1=1 



rxi/(ti) 



[tl ' T1 l^f^W^ + Cr 



= E 1 ^eO i a(ii,^,P i ,«| [(i , T] ) + Cr 
i=i 

Note that, if /xf € Oj, then ||/ t "'I T — < ??, so that, by the choice of 77, 

we get 

a(ti,Mi,-Pi,«| [tl .n)^ v+ (* 1 '^) + e / 2 - 

Therefore, recalling the definition of ^ and noticing that d(/^, /•4°' Me '' P ' E ' i; ) < 
Cr , we get 

3(to, Mo «) < V + (t 1; M ?) + e + Cr < V+(ti, ^ p ^) + e + Cr. 
Now, since P e is Ce— optimal for W(to,ti, fi e ) we obtain 

V + (t ,/u e )< sup 3(t ,A* e ,P,t;) < W(t ,ti,Me) + C(e + r) . 

Using again the fact that V + and W are Lipschitz continuous we have, as e 
and r are is arbitrary: 

V + (to,no) < Wfat^no) ■ 

Now we have to prove that the result still holds for measures with un- 
bounded support. Let G W. For all e > 0, there exists some closed ball 
K € centered at such that 

/ \x\ 2 dno(x) < e 2 and fj, (JR N \K e ) < e 2 . 
Ju N \K e 

Let T : JR N — > K w be such that T(x) = x for all x e K e and T(x) = for 
all x $l K e and let us set fj, t = TftfiQ. Then fi t G W(.fQ) and d(/xo,/x £ ) < e. 
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Let us now check that for all (P,v) G A(A x {t )) x V(t ), Mi := i i t t \^°' P ' v is 
close to /uf := ^ t t ° i ,fl€,P ' v . Indeed we have: 



Jm, N \K € JW(t ) ^W(to) 

< / 2[|x| 2 + 4(t 1 -t ) 2 ||/|| 2 oo]^o(x) < Ce 2 
Jtr n \k € 

Then the Lipschitz continuity of the upper value leads to: 

V + (t ,/io) < V+(t ,/x £ ) + Ce 

< inf sup V+fti./^'^'^ + Ce 
P6A(^(t ))„ 6V(to) 41 

< inf sup V + (ti,/4° , ' 10,p ' t ') + C , e. 
PeA(A c (to))„gv(to) 

Hence V + (io,^o) < W(to,ti, Ho) as e is arbitrary. 
We now prove that 

V + (to,no) > Wfatuno) . 



dno(x) 



(4) 



Let P be an e-optimal strategy for player I for V + (to,^o)- Let us fix vo G 
V(t ) and set m = ^^'^ . For all v G V(h), we define the measure P v 
on JR N x W(ti) by 



/ (p(x,u 2 )dP v (x,u 2 ) = \ 

JTR N xU(t!) Js 



K xW(to) 



> ; (< 0,x *"' ,;o ^i [tl , T] )^ ollto ' tl, ' ,;) w^ 



for any v G V(ii) and any nonnegative Borel measurable function 99 : 
E w x W(ti) -> JR. We note that the first marginal of P v is fJ ^ i ' IM> ' p ' Vo . 
Since is a Polish space, we can desintegrate P v with respect to 

^ ,p,v _ d pv( x ^ u ^ = dP^{u)d^°' Pm {x), where the mapping (x,v) -)■ 
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P% is measurable. Then P belongs to A(A x {ti)) and we have: 
V + (t 1 ,$ 1 ' IM, ' p ' V0 ) < sup f g{X^ mv )dP^{u 2 )d^' P ^{x) 

< sup / 5 X r 1 dP^ ((ui,ti 2 ))(l/io(i) 
u6V(ti) ^R^xW^ctOxW^i) V / 

< sup / fffx; '^'^ 0114 - 411 ^)^ ' 140 '' 11 '^^)^^) 

veV(ti) JlR N xU(to) v ' 

< V + (t ,fi ) + e 

Hence inequality (4) holds since vq and e are arbitrary. □ 

4 Characterization of the upper value function 

We prove in this section that if a function satisfies the previous dynamic 
programming principle, then it is the unique viscosity solution of some 
Hamilton-Jacobi equation. 

We consider the Hamiltonian H, defined for any fi G W and for any 
peLl(JR N ,JR N ),by 

H(/j,,p) = sup / inf / (f(x,u,v),p(x))du(u)dv(v)dn(x) , (5) 

veA(V) JlR^ ueA(C7) Jt/xy 

where A(C7) and A(V) denote the sets of Borel probability measures on the 
compact sets U and V respectively. Let V : [0, T] x W — >■ IR be a Lipschitz 
continuous map. We say V is a subsolution to 

Vt + H^D^V) = in[0,T)xW (6) 

if, for any test function /x) of the form 

(where ip : IR — > IR is smooth, a, rj > and /Z, u G W) such that V — </? has 
a local maximum at (P, i) G [0, T) x W and for any optimal transport plan 
7f G n opt (/x, P), one has 

ip'(t) + -op) > 
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where p is the unique element of Lp(5R N , M N ) associated to ir such that 



(see [3]). In the same way, we say V is a supersolution to (6) if, for any test 
function <p(t, fj,) of the form 



(where ip : IR — >■ IR is smooth, a, r] > and fx, v G W) such that V — <p has 
a local minimum at (is, t) G [0, T) x W, one has 



Proposition 4.1 (Comparison principle). Let w\ be a Lipschitz continuous 
subsolution of (6) and w 2 be a Lipschitz continuous supersolution such that 
w\(T,fi) < W2(T,(i) for any fj, £ W. Then wi < u>2 in [0,T] x W. 

In particular, given a Lipschitz continuous terminal condition g : W 
IR, the Hamilton-Jacobi equation (6) has at most one Lipschitz continuous 
solution V which satisfies V(T, n) = g(n) for any /iEW. 

Note that other definitions of viscosity solution in the space W have 
been introducted recently: see for instance [6, 7, 9, 10]. Our definition is 
closely related to the one of [3], which seems more appropriate for the kind 
of problem we have to handle. 

Proof. The proof borrows its main arguments from [4] , and follows closely 
[3]. We denote by K the common Lipschitz constant of w\, u>2 and /. 
Without loss of generality we can assume that 





(7) 



i>'(t) + H(i?,ap) < H/llooT?. 



new 



inf w 2 (T,fi) —wi(T,fi) = . 



(8) 



Our aim is to prove that 



inf 



w 2 (t,fi) - wi(t,[i) = . 



Assume on the contrary that 



inf 

(t,rie[o,r]xW 



w 2 {t,n) - wi(t,fi) = -£ < . 
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Fix (to, no) such that (u>2 — wi){to,Po) < — £/2- Denote by 

ip ev (s, fi, t, v) = w 2 (t, v) - wi(s, h) + ^-d 2 ^, v) + -(t- s) 2 - f]S . 

The function (p erj is continuous and bounded from below. Using some mod- 
ified version of Ekeland's variational Lemma (Lemma 6.4 below), we have 
that, for all 5 > 0, there is (s, p, t, v) such that for all (s, ji, t, v): 

<Per){s, P, t, v) < (f eri (t Q , fl , t , Ho) ,gs 

ip ev (s,p,t,v) < (p €V (s,n,t,v) + S[d(fj,,fi) + d(u,u)] 

Let us fix i G H opt (i',p). We first give a bound on the distance between 
(s, p) and (i, v). Since 

(p er ,{s, p, t, v) < (p er ,{s, p, S, p) + 5d(p, u) , 

we have 

w 2 (s, p) - u>i (s, p) - r/s + 5d(p, T>) 

> w 2 {i, v) - wi(s, p) + ^-d 2 (p, u) + i(t - s) 2 - i]s 

> W2(s, p) — K\s — i | — Kd(y, p) — wi(s, p) H — d 2 (p, v) -\ — it — s) 2 — rjs , 
that reduces to 

d(v,p) + \i-s\ <2e(K + S) . (10) 

We now seek some contradiction assuming that t,s ^ T. We first use the 
fact that 

tp t ri(s, p, t, V) < ip ev (s, fl, t, v) + <5d(/X, p) , 

namely 

W2{i,v) - w\(s,p) + ^-d 2 (p, v) + -^(i - s) 2 - ijs 

< W2(i, v) - wi(s, h) + -d 2 (n, v) + -(i - s) 2 - r/s + 5d(n,p) , 
leading to 

wi(s, n) — -d 2 (p,, v)— 5d(p,, p) — -(t—s) 2 +r]s < w\(s, p) — -d 2 (p, L>) — -(t—s) 2 +r]s . 

If we set ip(s,ii) = ^-d 2 (/i, U) + 5d(n,p) + \(t — s) 2 — i]s, then the function 
w\ — (f has a maximum at (s, p). The function w\ being a subsolution, we 
get by definition: 

-V ~ s) + H{p, --v) > -(5H/IU (11) 
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where p is defined by: 

/ (Z(y),x-y)dir(x,y)= [ Uy)My))dp{y) V£ G Ll(JR N ,M N ) . 
The same argument applied to 

ip erl (,S, p, t, V) < ip ev (s, p, t, v) + 5&{v, u) 

leads to 

-l{t-7s) + H{v,-^)<5\\f\\ 00 (12) 

where q satisfies 

/ (ay),x-y)d7t(x,y)= [ (e(x),q(x))di/(x) V£ G L 2 P (M N , JR N ) , 
7f being defined by 

/ y)d7f (x, y) = / <p(y, x)d7r(x, y) G L 2 (IR 2iV , IR 27V ) . 

Note that 

/ (ax),x-y)d7r(x,y) = [ (ax),-<l(x)}dv(y) V£ G L 2 ^, M N ) . 
Jn 2N Jm n 

Combining (11) and (12) we get 

n + H{u, Hq) _ H(p, -Hp) < 2^11/Hoo . (13) 

Let us now recall some continuity property of the Hamiltonian H defined 
by (5): 

Lemma 4.2. Let {p,p) G W 2 and (p,q) G L?^, M N ) x ^(E^.IR^) 6e 
suc/i that, for some it G Il op t(i',p), 

[ (ay),x-y)d7r(x,y)= [ (ay),p(y))dp(y) V£ e LlQR N ,TR N ) 

and 

[ (ax),x-y)d7r(x,y) = [ (e(x),-q(x))di/(x) V? G Ll(M N ,M N ) . 
Jn 2N Jtr n 

Then we have: 

\H(p,p)-H(i?,-c l )\<Kd 2 (i?,p) 
where K stands for the Lipschitz constant of the dynamics. 
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Proof. The proof is the same as in [3], Lemma 6. □ 
Therefore, we have: 

2 2 2K o 

|ff(/2,--p)-fl-(P,-q 1/ )| < — d 2 (^,/2) • 

Thus using the previous inequality and estimate (10) in (13), we get 

r/<2<5||/|| 0O + 8^e(^ + ( 5) 2 
leading to a contradiction for e, 5 sufficiently small. 

This implies that we have t = T or s = T. Assume for example that 
s = T. We have 

ip er ,(s,p,,i,v) < ip er) {to, fio,t , fio) < -£/2 . 

Therefore using (8) and (9) we obtain: 

-e/2 > w 2 (t, v) - Wl (T, j2) + ^d 2 (/i, v) + \t- if - r)T 

> —K\T - t\ - Kd(v, ft) + ^d 2 (fi, v) + \t- t) 2 - r]T 

> - K [\t - t\ + d{fl, u)] + ^[\T- l \ + p )] 2 " 
Using (10), we finally get: 

f/2 < 2e(K + 5)(2K + S)+r}T 

which is impossible for e and r\ small enough. □ 

Proposition 4.3. The upper value function V + is the unique Lipschitz 
continuous viscosity solution of the Hamilton- Jacobi equation (6) satisfying 
the terminal condition: 

V+(T, / u)= / g(x)d»(x) (14) 

Proof. We only prove that V + is some solution, uniqueness being an obvious 
consequence of Proposition 4.1. Let us recall that V + satisfies the dynamic 
programming principle 

V + (t ,u)= inf sup \ + {h + h,^'lf ,V ) (15) 
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where /4°+/i P '" is the measure defined by 



IR xU(to) 



Let us show that V + is a subsolution. Let ip = <p(t, fi) be a test function of 
the form 



a 



<p(t, n) = ^d 2 (/i, (i) + rjd(u, n) + fP(t) 

(where ip : IR — > IR is smooth, a, rj > and /}, P G W), such that V + — 99 
has a local maximum at (i>,to). Without loss of generality we assume that 
<p(to, v) = V + (to, v). Then V + < 92. We fix an optimal plan tt € n op i(/2, zv). 
From (15), we get 



< inf sup 

PeA(A x (t )) ve v(t ) 



p(t + h, $ Q '+£' V ) - (p(to,u) 



(16) 



Setting for simplicity i/(to + h) = \i^+^ v and recalling the definition of 92 
we have 



<p(to+h, v(t +h))-<p(t , v) = - [d 2 (/Z, v{t Q + h)) - d 2 (/2, u)] +??d(^, v(t +h))+ip(t +h)-i(j{to) 

(17) 

where 



d(v,v(to + h)) < 



f 

JjR N > 



y ^t +h 



Recalling the definition of p in (7), we also have 
d 2 (j2,u(t + h)) 



dpv(u)di?(y) 



<\\f\\ooh. 



_ _ vto,y,u,v 
x ^t +h 



dP£(u)djt(x,y) 
rto+h 



if 

J]R 2N xU 

Jn 2N [Ju(to) Jto 

r r rto+h 

<d 2 (^v)-2 (p(y), / / f( y , U (s),v(s))ds dPV(u) 

JTR N Ju(t Q ) Jt 



u 



}dTt(x,y) + Ch 2 



)dv{y) + Ch 2 



Let (U n ) be an increasing family of finite subsets of U such that (J n U n 
is dense in U and U n be the set of Borel measurable maps x — > u x from 
E w into A (U n ). The main point in this discretization is that A(U n ) is a 
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compact subset of some finite dimensional space. Therefore U n , endowed 
with the weak topology of L?, is convex and compact. Since U n can be 
viewed as a subset of U(to), one can associate with a map u € U n a strategy 
P u £ A(A x (to)) defined by the equality 

/ ip(x,u)dP^ = / ip(x, u)dxY x {u)di>(x) 

JM N xU(t ) JTR N xU n 

for any v £ V(to) and any nonnegative Borel measurable map <p : TR N x 
U(t ) -> IR. Recalling (16) and (17), we get 

0<^(t ) + \\f\\ooV + Ch 

+a inf sup / (-p(y), / -/ f(y,u,v(s))dsdu y (u)dQ(v))d&(y). 

ueU " QeA(v(t )) ■/ni JV Ju n xV(t ) h J t 

By Sion's min-max Theorem we get 

inf sup / (-p(y), / 7-/ f{y,u,v{s))ds dwy(u)dQ(v))dv{y) 

ueU ™ QeA(V(t )) JR W JU n xV(to) h Jto 

r f 1 [ to + h 

sup inf / (-p(y), / t / f(y,u,v(s))ds du y (u)dQ(v))di>(y) 

Q6A(V(to)) ueU -m* Ju n xV(t ) h Jt 

= sup / inf (-p(y), / 7-/ }{y,u,v(s))ds du(u)dQ{v))dv(y) 

QeA(V(t )) JR" u eA(l/„) Ju n xV(t ) h Jt 

< sup / inf ess-sup 86[to t0+h] (-p(j/), / /(y, du(u)dQ(v))di/(3/) 

< sup / inf (-p(y), / f(y,u,v) du(u)dv(v)}dv{y) 
veA(v) JiR^ ueA(f/ n ) Jc/ n xy 

So 

< ^'(t ) + a sup / inf {-p(y), [ f(y,u,v)du(u)dv(v))dv(y) + \\f\\ 00 r ] + Ch 
veA(V) J TR N u £ A (Un) Ju n xV 

Letting /i — > and n — > +00 gives the desired inequality since |J n A(t/ n ) is 
dense in A ([/"). 

We now check that V + is a supersolution. Let ip = ip(t, fi) be a test 
function of the form 

M) = ~f d2 (^' ^) - ^ d (^' ^) + ^(*) 

(where ^ : IR — > ]R is smooth, a, 77 > and //, £ W), such that V + — 99 
has a local minimum at (u, to). We again assume that (p(to, 9) = V + (io, v), 
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so that V + > if. Let us apply the dynamic programming at time to and for 
v. We get 



> inf sup 

PeA(A x (t )) veV(t ) 



Setting as before v(to + h) = fi t t 0, ^' v , we have, for any P £ /\{A x {to)) and 
«GV(t ), 

d 2 (fl,u(t +h)) < d 2 (fi,»)-2 / (p(y), / / f(y,u(s),v(s))dsdP^(u) 



)dv{y)-Ch 2 



so that 

> ^-H/Iloo^-C/J 



inf 



sup 



PeA(A x (t )) QgA(V(t )) JK f 



/ (p(y), / 7- / /(v, 

JlR^ JU(t )xV(t ) n Jto 



u(s),v(s))ds dPV{u)dQ(v) 



Let P/j G A (,4a; (to)) be /i— optimal in the above expression. We denote by 
Th its delay and set nh = h/rh- Reducing th if necessary, we can suppose 
that rih is a positive integer. Let us set tk = to + /cr^ for k = 0, ... , rih- Let 
us now fix v € A(V). With v we associate the strategy Qh,v consisting in 
choosing randomly, on each time interval [tk, tk+i] (where k = 0, . . . , rih — 1) 
a time independant control v according to the probability v. We now claim 
that 



/ 

Ju 



t +h 



<p(y), / f(y,u(s),v(s))ds) dPl y {u)dQ Kv {v) 
W(to)xV(t ) Jto 

- h in L / (P(y),f(y,u,v))du(u)dv(v) 
ueA(u)J UxV 



(19) 



for v— a.e. y. For this it is enough to show by backward induction on 
k € {0, . . . , rih} that 



r rto+h 

/ <p(y), / f(y,u(s),v(s))ds) dPl y {u)dQ htV {v) 

Ju(t )xV(to) Jt 

> I (P(y), / " /(y,«(s),«(s))da) dP^{u)dQ 

Ju(t ,t k )xV(t ,t k ) Jt 

+ {n h -k)T h inf / (p(y), /(y,u,t;)) du(u)dv(u) 



(20) 
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where P^ and Q\ v are denned as the restriction of the strategies Ph and 
Qh,v to the time interval [to, to + kr^]- Note that the above inequality is 
obvious for k = rih- Let us assume that it holds for k + 1 and prove that it 
still hold for k. We use the decomposition 

V(to,tk+i) = V(to,t k ) x V(t k ,t k+1 ) 

and write v = {v\,v 2 ) for any t> G V(fojife+i), where v\ € V(io,ifc) and 
v 2 € V(tfc,tfc + i). By definition of Q** 1 , we have 



X 



■/Vftn.titlxV 



v(t(),t fc+ i) ^v(to,t fc )xy 

Therefore, since P has a delay we get, omitting the arguments of p and 
/ for simplicity 

/ (p, f k+1 fds)dP^(u)dQ k + / 1 (v) 

JU(t ,t k+1 )xV(t ,t k+1 ) Jt 

(p, f k+1 fds) d^v 2 )dP k h ^{u)dQl v { Vl ) 
IU(t ,t k+1 )xV(t ,t k )xV Jt 

(p, f k fds)dP^(u)dQl v ( Vl ) 
IU(t ,t k )xV(t Q ,t k ) Jt 



> 



+Th inf / (p, fds)du(u)dv(v ) 
ueA(u) J UxV 



because 

ftfc+1 



[ (p, / "'" f{y,u{t),V2)ds) dv{v 2 )dP k h + y l ^(u)dQl v ( Vl ) 

JU(t ,t k+1 )xV(t ,t k )xV Jt k 

> inf t „ / (P, / f(y,u(t),v 2 )ds) dv(v 2 )dP{u) 

PeA(u(t k ,t k+1 )) Ju(t k ,t k+1 )xv Jt k 

= T h inf / (p, / f(y,u,v 2 )ds) du(u)dv(v 2 ) 
ueA(u)J UxV J tk 

where A(U(tk,tk+i)) stands for the Borel probability measures on the set 
lA{t k ,t k+ \). This gives (20) by induction. 
Combining (18) with (19) we get 

> ^(t ) - \\f\\oor]-Ch+a [ inf / <p(y), f(y, u,v)) du(u)dv(v)d»(y) , 

J u n ueA(U) J UxV 

and we obtain the desired inequality by letting h — > 0, since v is arbitrary. 

□ 
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5 The discretized game 



In order to prove that the game has a value, we have to introduce some aux- 
iliary discretized game for which the existence of the value can be obtained 
by classical min-max arguments. 

5.1 Discrete strategies of Player II 

In this new game, the actions of Player II are random controls defined on a 
suitable finite set. More precisely, let us fix an integer n > 1. Let r n = — be 
the time step and tf = ir n (for i = 0, . . . , n) be a grid on [0, T]. We consider 
an increasing family (V n ) of finite subsets of V such that, for any n > 1 and 
any v G V, there is some v n G V n with \v — v n \ < 1/n. For each n > 1 and 
to G [Oj^Ij we denote by V n (to) the finite subset of V(io) consisting in step 
functions with constant value on each interval [tf,tf +1 ) and taking values 
in V n . Let A(V n (io)) be the set of all probability measures over V n (to). 
If we denote by N n the cardinal of V n (to), then the set A(V ra (io)) can be 
identified with the simplex of TR Nn because each element of A(V n (to)) can 
be written as v = Y^=iPi v i f° r some p G (IR + ) Arn such that p ■ 1 = 1. Then 
we set d?{v,&) = d^^pm^hP^i) = \\p ~ P'h- The set A(V n (i )) 
can therefore be viewed as a compact, convex subset of JR Nn . 

5.2 Discretized game 

The discretized game is the game where Player I plays some strategy P £ 
A(^ n (to)) and Player II plays some random control v £ A(V n (to))- Our 
aim is to use Sion's Theorem in order to prove that the discretized game has 
a value. For this we see A(A x (to)) as a convex subset of the vector space of 
the set of maps from IR^ x V(io) into the set of Borel signed measures on 
U(to). We can endowed A(A x (to)) with the distance 

d(P,P)= sup d Amt0 ))(PZ,PZ) 

(x,v)eR N xV(t ) 

(recall that the distance ^A(w(t )) on ^(^(^o)) is defined in section 1). For 
any fixed (to, Mo) G [0, T] x W, we note that the map (P, Q) — > 3(to, P, Q) 
is linear with respect to P and to Q and continuous with respect to both 
variables on A(^ n (to)) and A(V n (to))- Indeed the continuity with respect 
to Q is obvious since A(V n (to)) is finite dimensional. The continuity with 
respect to P also holds because, since the map u — > g(X^' x ' u,v ) is con- 
tinuous on U(to) for any (x,v), the map p — ¥ J u < t g(X^' x ' u,v )dp(u) is 
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continuous on A(U(to)) for any (x,v). The continuity of the map P — > 
Iu N xU(t )xV(to)9( X T' X ' U,V ) dP x(u)dQ(v)dfi (x) on A(A x (t )) then follows 
from Lebesgue dominate convergence Theorem since g is bounded. 
We can now use Sion's minmax Theorem to get: 

Lemma 5.1. For alln G IN*, the discretized game on A(^ n (^o))xA(V n (to)) 
has a value, denoted by V n (io, Mo)-' 

V n (i ,Mo) = hif sup / 5 (X^ ,x,u ' v ) dP^(u)dQ(v)d/j, (x) 

PGA(^"(i ))Q e A(V n (io))^]R Jv xW(to)xV n (to) V y 

inf sup f g(x^ u ' v )dP^(u)dMx) 

PeA(Al"(to)) veVn (t )JTR N xU(t () ) V / 

= sup inf / 5 (x£>' w ) dPV(u)dQ(v)dfi (x) 

QeA(V n (to))P^(Al n (to))JsR N xU(t )xV n (t () ) V 7 

= sup inf / g (X^' x ' a{x ' v) ' v ) dQ(v)dfi {x) 

QeA(V„(t )) ae - 4 J n (*«)-'lR Jv xV Il (to) V 7 

Moreover V n is Lipschitz continuous in both variables uniformly with respect 
to n. 

Proof. The Lipschitz continuity of V n can be established as in Proposition 
2.1. □ 

Recalling the definition of V r in (3) one easily gets: 
V n (i ,Mo) < V~ (t , Mo) • 

It remains to check that liminf n _ i , 00 V n (to, Mo) > V + (io,Mo)- This is the 
aim of the next section. For this we need two preliminary lemmas: 

Lemma 5.2. The value function V n satisfies the dynamic programming 
principle: 

V n (tl»)= inf sup V^ +1 ,4;^). 

PeA(^"(t»))„ eVn (ijj) fc+1 

Proof. The proof is closely related to that of Proposition 3.1, so we only 
explain the main differences. Let us denote by W(t^,fi) the right-hand side 
of the above equality. One can check, as in the proof of Proposition 2.1, 
that W is Lipschitz continuous with respect to fi. Inequality V ra > W can 
be established as in Proposition 3.1. 
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Let us now prove the inequality V n < W. Let e > and Pq be e— optimal 
for W(t%,fj,). Let us fix 5 > small and v G V n (i£). We set s = tj? and 
si = ife+i- At time si + 5, Player I knows which constant control Vi £ V n 
Player II has been playing on the time interval [sq, si], so he or she knows 
the measure vi := /4°' /i ' Po '' y \ Let Pj be e— optimal for V n (si, fj). We restrict 
the strategy p to the time interval [s\ + S, T] by setting 

f ^u 2 )dPl x {u 2 ) = ! ^ ))dP^ T] (u) 

for any (x, t>) £ ffi/^ x V(so) and any nonnegative Borel measurable map 
(p : U(s\ + S) — s> IR. We finally define the strategy P € A^-^so)) by using 
the identification U(sq) = U(sq, s\ + 5) x li(s\ + 5): 

dPV(( Ul ,u 2 )) = dPgJm) f ^ 1 { } dP" PoiVi (u 2 ) 

This means that Player I plays the strategy Pq on the time interval [sq, si+S], 
and then switches at time si + S to the strategy Pj evaluated at the point 
^so,x,p ,t), ■£ ^ e con ^ ro j pi a y eo ; D y Player II on [so, s±] has been Vi. It is then 

a routine computation to show that the strategy P satisfies 

3(s , A* e , ^ *0 < ^( s o, M) + Ce + 0(1) , 

where O(l) — > as 5 — > 0, uniformly with respect to v and we conclude as 
for Proposition 3.1 that V n < W. □ 

Lemma 5.3. The value V n of the discretized game satisfies for all test 
function 

'Pit, M) = _ ^" d2 ^' ^) - fj) + ^(*) 

(where ip : IR — >■ IR is smooth, a, n > and p,,i> &W) such that V n — 99 /ias 
a global minimum at (tJJ, P) and /or any optimal plan it £ H opt (fi, v): 

> V'(^) + «P) -WfWooV-O , (21) 

where p is defined by 

[ (Z(y),x-y)dir(x,y)= [ (Z(y)Mv))dHv) ^ e Ll(M N ,M N ) 

and where the Hamiltonian H n is given by: 

H n (ji,p)= max / min / (p, f(y, u, v))du(u)dv(v) dfi(y). (22) 
veA(p) J m n ueA(u) J UxV 
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Proof. We assume, without loss of generality, that V n (t k ,u) = ip(t k ,u). 
Applying the dynamic programming principle of Lemma 5.1 we have 

> inf sup + r n , u{t n k + r n )) - <p(ft, u)] 

where we have set 

Using the special form of ip, we get: 



a 



> inf sup + r n ) - V(^) + ^[d 2 (/i, P) - d\fi, v(t n k + r n ))] - nd(u, v(t n k + r n ) 

Arguing as in Section 4 we have: 

K>T n ^{tl)-0{T 2 n ) 



+ inf sup 



a 

M N xU(t»)xV(t%) 



(p(y), / f(y,u(s),v(s))ds)dP^(u)dQ(v)du(y)- V \\f\\ 00 r, 



We now note that the restriction of the strategy Q to [t k ,t k + r] just con- 
sists in playing a constant control at random according to some probability 
measure v <G A(V n ). Moreover the strategy P, having for delay r n , does 
not depend on v on this time interval and therefore amounts to playing at 
random a control u with probability P x (u) independent of v. Denoting by 
A(U '(<£))) the set of probability measures on U(t k ) and using the min-max 
Theorem, we have 



0>r n ^(^)-^ll/lloor n -O(^) 

+ q max / inf / (p(y), / + f(y,u{s),v)ds)dP{u)dv(v) dv{y) 

veA(v„) J m n PeA(w(t™)) J U (q)xV Jt" 

>r n ^(tD-^ll/llooT n -0(r n 2 ) 

+ «r n max / min / (p(y),f(y,u,v))du(u)dv(v)dv(y). 
veA(v n ) Jirn ueA(u) J UxV 

□ 

6 Existence and Characterization of the value 

Theorem 6.1. The game has a value, namely: 

V+ = v 
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characterized as the unique viscosity solution of the Hamilton- J acobi equa- 
tion (6). 

Proof. We have already noticed that V n (io,Mo) < V^(tO)Mo)- It remains 
to check that liminfV^oo V n (io, A*o) > V + (to,/io)- The main idea is to use 
the stability of viscosity solutions as in [5]. From Proposition 4.3 we know- 
that V + is a solution of the Hamilton- Jacobi equation (6), while Lemma 5.3 
states that V„ is "almost a subsolution" of that equation. Moreover, the 
functions V n and V + are bounded and Lipschitz continuous with the same 
Lipschitz constant denoted by K and we have V + (T, fi) = V n (T, fi) for all 
new. 

Let us introduce the functions U n (t,/x) = e*V n (t,/x) and U + (i,/i) = 
e'V + (t,/i). The function U + is a viscosity solution of 

-V + XJ t + H(ji,D li U) = (23) 

with final condition U + (T, fx) = e T J m n g(x)dfx(x). For (s, //), (t, i/) G [0, T] x 
W, with s £ {tf, i = 0, . . . , n}, we set 

m, t, f) = u+(t, i/) - u„( s , n) - -d 2 {n, u)--(t- sf . 

e e 

The function y?" is continuous and bounded from above. From Lemma 6.4, 
for all 5 > 0, there exists (s, p,, t, v) 

(p?(s,ji,i,v) > <p?(s,iJi,t,v) -S[d(n,p) + d(v,v)] V(s,/i,t,i/) (24) 

and 

(p?(s,p,,t,i>)> sup ip^(s,n,t,v) - S . 

(s,n,t,v) 

This auxiliary function gives a bound on sup( t>M )[V + (i, ll) — V n (i, /z)]: 

sup[V + (i,/z) - V n (t,/i)] < sup[U + (i,/x) -U„(t,/i)] < <p?(8,ii,i,P) + 5 . 
(t,/i) (t,/i) 

(25) 

Now we use the fact that 

(«, Mi *, ^) > (*. A, «, A) - ^ d (A, V) , 

namely 

u+ (s, //) - U„(s, //) - <5d(/Z, P) 

< U+(t, P) - U n (s, A) - ^d 2 (/2, P) - - s) 2 

< U + (s,fi) + K\s-i\ + Kd(v,fi)-XJ n (s,j2) - -d 2 (ji,v) - -(it- s) 2 , 
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to get the following bound on the distance between (t, v) and (s, /j,): 

d(v,p,) + \i-s\ < 2e(K + 6) . (26) 

Let us first assume that s = T (the case i = T could be treated similarly). 
Then we have: 

tf(8, fi, i, u) < U+(T, P) - U n (T, ft) + K\T - i\ 

<e T g(x)du{x) - e T / g(x)dp(x) + KIT - i\ 
Ju N Jn N 

< Lip(c/)e T d(P,/2) +K\T-i\ < Ce 
Thus using (25) we get: 

sup[V+(t, - V n (t, m)] < + 5 , 

so that, passing to the limit as e, 5 — > 0, we obtain: 

sup[V + (t,/i)-V n (*,/i)] <0. 

(*,/*) 

We now assume that i ^ T and s ^ T. Let us fix some optimal transport 
plan 7r G Uop t (i> , ft) . We first use the fact that 

ip e (s, fi, i, v) > ip e (s, \i, i, v) - Sd(n, p) , 

namely 

U+(t, P) - U„(I, ft) - -d\ji, u) - l -{t- sf 

> U+(i» - U n (s,n) - ^d 2 (n,u) - -(*- s) 2 - 5d(/i,/Z) , 

to get 

1 1 _ 11- 

V n (s,n) + -d 2 (fi,u)+5d(fi,p) + -(i-s) 2 >V n (s,p) + -d 2 (fi,u) + -(t-s) 2 . 

If we set <p(s,[i) = — \d 2 {n,v) — <5d(/x,/Z) — ^(i — s) 2 , then the function 
U n — if has a minimum at (s,p). From Lemma (5.3), this implies that: 

-U n (a,/2) + - «) + fln(A, ^P) < *ll/l|oo + O 
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where p is defined by: 

/ (Z{y),x-y)dK{x,y)= [ Uy),v{y)W{y) Vt£Ll(JR N ,M N ). 
Jn 2N Jtr n 

(27) 

The same argument applied to 

<Pe (*> £> t, v) > ip™(s, p,, t, v) - 5d(v, v) 

leads to 

u+(t, p) _ _ i) _ -Hq) < ^ii/iu 

where q satisfies for tt € n opt (/x, p): 

/ ^(y),x-y)d7f(a;,y)= / (£(*), q(x))d9(x) V£ G L 2 P (M N , JR N ) . 
If we take for 7f the optimal transport plan defined through: 

/ <p(x,y)dir(x,y) = f y>(y,x)dTr(x,y) V<p £ Ll(M 2N ,JR 2N ) 
we have: 

/ (Z(y),x-y)dir(x,y)= f (^x),-q(x))dP(x) ^eL 2 p (M N ,M N ). 
Jtr 2N Jtr n 

Finally combining the last two inequalities we obtain: 

U+(i,v)-XJ n (s,fi) <H(u,-\)- H n {fl, -v) + 28\\f\\ O0 + o(^j • (28) 

Our next step consists in comparing H and H n : 

Lemma 6.2. We have, for any fi € W and any p € L^(TR N ,TR N ): 

< H(p,p) - H n (ji,p) < 7n||p||Lj , 

where 

7n = SUp \f(x,U,V 1 )-f(x,U,V 2 )\. 

x,u,\vi—V2\<l/n 

Remark 6.3. Note that 7„ — > as n — > +oo because / is uniformly con- 
tinuous on M N x U xV. 
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Proof. By definition we have H(ft,p) > H n (/j,,p). Let v be e— optimal for 
H(fi, p) and II : V — > V n be a Borel measurable selection of the projection 
map from V onto V n . Then, by construction of V n , we have \v — n(u)| < 1/n 
and 

H„(fi,p) < inf / (/(x,n,v),p(a;)}du('u)cffljjv(v) d/x(x) 

7 R iv ueA(U)J UxV 

< inf / (f(x,u,v),p(x))du(u)dv(v) dfj,(x) + j n \p(x)\d/j.(x) 
J m n ueA(u) J UxV J m n 

< H(fi,p) + e + 7 n ||p|| L 2 

□ 

From (27), we have 

||p||i2 < d(fi,v) . 

Combining the continuity of the Hamiltonian stated in Lemma 4.2, the 
bound on the distance between H and H n given in Lemma 6.2 and inequality 
(28) we get: 

U+(t, P) - U n ( S - ft) < —d 2 (u, ft) + ^d(u, ft) + 25H/IU + O ( - ) . 

Putting together the estimates (25) and (26) and letting e, 5 — > finally 
gives 

v+ < v n + c ln + o . 

This implies that liminf^oo V n (io, Ho) > V + (to,^o) an d completes the 
proof. □ 



Appendix 

The following statement is a slight modification of Ekeland's variational 
Lemma. 

Lemma 6.4. Let F : [0, T] x W — > 1R 6e a continuous function which is 
bounded from below. Then for any e > 0, there exists (i, ft) & X such that 
for all (t,fi) 6 X: 

F(t,/j,)>F(i,ft)-ed(n,fi) and F(t, fi) < inf F + e . 
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Proof. Let (ioiMo) be such that 

F(to,no) < infF + e . 

X 

Then we build the sequence (t n , p n ) by induction, such that, if (t n ,fj, n ) is 
known, then 

• if for all (t, jj) € X, F(t,/j.) > F(t n ,/j, n ) — ed(/x, /x n ), then we set 

{t n +l, Hn+l) = (t n ,[i n ), 

• if, on the contrary, there is (t, /x) G X such that F(t,p) < F(t n ,/J, n ) — 
ed(fx, (j, n ), then we set 

S„ = {(<,//) € X such that F(t,fi) < F(t n ,fi n ) - ed(//,/i n ) } 

and choose Mn+i) G S n such that F(i n+ i,/z n+ i) < (F(t n ,fx n ) + 

inf Sn F)/2. 

Note that by construction F(t n , // n ) < infx + e for any n. We prove that 
the sequence (/i n ) is & Cauchy sequence. Indeed, it is either stationary, or 
we have ed(^ n ,^, n+ i) < F(t n ,fi n ) - F(t n+ i, fi n+1 ). Therefore for all (n,p), 
n > p, 

ed(/i n , Hp) < F(t p , Hp) - F(t n ,Hn) . (29) 

The sequence F(t n ,p n ) being decreasing and bounded from below, it has a 
limit and inequality (29) shows that (//„) is a Cauchy sequence. Let p, be 
the limit of the (fj, n ) and let us consider any cluster point t of the (t n ). We 
now assume for a while that the is some (s, v) € X with 

F(s, v) < F(i, p) - ed(v, p) . (30) 

Consider some subsequence (t^j MnJ converging to (t,p). Letting n — > +oo 
in (29) gives 

F(i,p) < F(t ni ,n ni ) - ed(nni,Jj) ■ 

Therefore, we have 

F(s,v) < F(t ni ,Hm) ~ ed(P,^ n J , 
which means that (s, v) G S ni for all i. This implies that 

2F(t ni+1 ,fi ni+ i) - F(t ni ,fi ni ) < inf F < F(s,») . 

The sequence F(t n , Hn) being decreasing, we get: 2F(t Hi+1 , fJ, ni+1 )-F(t ni , /x n J < 
F(s,iy). Passing to the limit as i — > oo gives F(t,p) < F(s,i>), which is in 
contradiction with (30). Therefore we have 

F(t,fx)>F(i,p)-ed(H,p) V(^)£l. 

□ 
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